Likelihood-ratio calibration using prior-weighted proper scoring rules
نویسندگان
چکیده
Prior-weighted logistic regression has become a standard tool for calibration in speaker recognition. Logistic regression is the optimization of the expected value of the logarithmic scoring rule. We generalize this via a parametric family of proper scoring rules. Our theoretical analysis shows how different members of this family induce different relative weightings over a spectrum of applications of which the decision thresholds range from low to high. Special attention is given to the interaction between prior weighting and proper scoring rule parameters. Experiments on NIST SRE’12 suggest that for applications with low false-alarm rate requirements, scoring rules tailored to emphasize higher score thresholds may give better accuracy than logistic regression.
منابع مشابه
The PAV algorithm optimizes binary proper scoring rules
There has been much recent interest in application of the pool-adjacent-violators (PAV) algorithm for the purpose of calibrating the probabilistic outputs of automatic pattern recognition and machine learning algorithms. Special cost functions, known as proper scoring rules form natural objective functions to judge the goodness of such calibration. We show that for binary pattern classifiers, t...
متن کاملPartial Likelihood-Based Scoring Rules for Evaluating Density Forecasts in Tails
We propose new scoring rules based on partial likelihood for assessing the predictive accuracy of competing density forecasts over a specific region of interest, such as the left tail in financial risk management. These scoring rules are proper and can be interpreted in terms of Kullback-Leibler divergence between weighted versions of the density forecast and the true density. Existing scoring ...
متن کاملWeighted likelihood ratio scores for evaluating forecast densities in tails
We propose and evaluate several new scoring rules based on likelihood ratios, for comparing forecast densities in the context of VaR modelling and expected loss estimation. Our approach is motivated by the observation that existing scoring rules tend to favour fat-tailed models when compared with thin-tailed models. Rather than restricting the weight functions, we impose some restrictions on th...
متن کاملWeighted Likelihood Ratio Scores for Evaluating Density Forecasts in Tails∗
We propose and evaluate several new scoring rules based on likelihood ratios, for comparing density forecasts in the context of VaR modelling and expected loss estimation. Our approach is motivated by the observation that some existing weighted scoring rules tend to favour fat-tailed predictive densities over thin-tailed predictive densities. Rather than restricting the weight functions, we imp...
متن کاملLikelihood-based scoring rules for comparing density forecasts in tails
We propose new scoring rules based on conditional and censored likelihood for assessing the predictive accuracy of competing density forecasts over a specific region of interest, such as the left tail in financial riskmanagement. These scoring rules can be interpreted in terms of Kullback–Leibler divergence between weighted versions of the density forecast and the true density. Existing scoring...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2013